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ABSTRACT 


We propose developing a method to set key educational skills which students need to achieve for each class using a 
student self-assessment questionnaire in analytical approach. It is difficult to set key academic skills for class since there 
are little systematic methods to set them. The questionnaire survey with 25 educational skills was conducted to ICT 
classes in our university using a computer-assisted web-interviewing (CAWJ) technique. The questionnaire results are 
analyzed using self-organizing maps (SOMs). A SOM is used to visualize the similarity relations between educational 
skills based on student's consciousness. We show that 25 skills were classified into several skill groups of introduction, 
fundamentals, and advanced courses, respectively. 
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1. INTRODUCTION 


The skills and attributes to be learned in school education are changing in the 21st century. There are various 
skills and attributes such as the 21st century skills announced by the United States Department of Education 
and the key competencies of the DeSeCo project organized by the OECD. Skill and attribute targets 
including the curricula of various countries are classified into the following three categories. The first is 
"pasic literacy" dealing with languages, mathematics and information, the second is "cognitive skills" dealing 
with thinking ability and learning way, and the third is "social skills" related to society, relationships with 
others and independence. In Japan, the National Institute for Educational Policy Research conducted research 
on curriculum which fostered attributes and skills. This institute conducted integrated and experimental study 
of educational goals, contents, method and evaluation. In future, it is necessary to advance research on 
learning method, teaching method and the evaluations. 

Tokai University to which we belong has formulated the following four key abilities as a specific 
evaluation indicator and training for cultivating social practical power since 2009. The first is thinking 
ability, the second is communication ability with others, the third is challenging ability and the fourth is 
accomplishment ability. The faculty has denoted the appropriate abilities and evaluation indicators in a 
syllabus as a skill to train in class. However, the skill setting method is not theoretical or systematic 
approach. The purpose of this study is to set effective educational skills and educational performance 
indicators for each class in analytical way. 

As a research to evaluate evaluation indexes and skills, there are meta-analyses (Robinson, et al, 2008; 
Poropat, 2009; Stajkovic and Luthans, 1998), data envelopment analyses (Avkrian, 2001; Abbott, 2003), and 
mediational analysis (Elliot, 1999). The meta-analyses examined the relative impact of different types of 
leadership on students’ academic and nonacademic outcomes (Robinson, 2008), and the analyses examined 
the relationship between self-efficacy work-related performance (Stajkovic, 1998). Poropat (2009) reported a 
meta-analysis of personality-academic performance relationship with cumulative sample sizes ranging to 
over 70,000. The data envelopment analyses examined the relative efficiency of Australian universities 
(Avkrian, 2001) and estimated technical and scale efficiency for the population of Australian universities 
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(Abbott, 2003). The study (Elliot, 1999) examined achievement goals, motivational study strategies and 
exam performance by a mediational analysis. 

We have examined the relationship between students' educational skills using a multidimensional scaling 
(Taniguchi, 2018). The multidimensional scaling (MDS) is a method of multivariate analysis. It is a method 
of summarizing the relationships of classification objects in a low dimensional space and visualizing the 
relation of classification objects. Cano (2002) informed an evaluation of Spanish educational research 
journals using multidimensional scaling. Pounder (2009) studied an evaluating the relevance of quality to 
institutional performance assessment. Gatewood (1993) studied the relationship between initial job choice 
decisions, the corporate images and recruitment images by multidimensional scaling. However, MDS 
solutions require a high dimensional space to explain the relevance of complicated data. 

In this study we survey the self-assessment questionnaire of students’ educational skills using a 
self-organizing map (SOM). SOM (Kohonen, 1995) is an unsupervised neural network method. It is suitable 
for analyzing complicated data and its calculation is not difficult. The SOM is an efficient tool for visualizing 
the relationship between multidimensional data. A SOM was used to visualize the similarity relations 
between educational skills based on student's consciousness. 


2. METHODS 


2.1 Participants 


A questionnaire survey was conducted in the beginning classes of the spring semester in 2018. 
A questionnaire for students of 19 ICT classes and 13 faculties was surveyed in Tokai University. The classes 
are classified by difficulty for introduction course, fundamentals course, and advanced course. Students can 
take these ICT classes regardless of student years or their faculties. Table 1 shows the student years, the 
genders, the faculties of survey respondents. All participants agreed to participate in the study. A cumulative 
total of 946 students participated in this study. Table 3 shows the list of ICT class names and the number of 
students. 


Table 1. Survey respondents and legend of symbols in Figures 1 to 3 


Symbol Introduction Fundamentals Advanced 

Total 578 273 95 
Year First 1 108 33 0 
Second 2 166 100 6 

Third 3 207 93 49 

Fourth 4 97 47 40 

Gender Female F 85 58 8 
Male M 493 215 87 

Faculty Letters LT 49 41 3 
Cultural and Social Studies Cc i: 2 0 

Political Science and Economics P 69 28 2 

Law L 22 13 2 

Humanities and Culture HC 13 9 1 

Physical Education G 20 6 1 

Health Studies H Z Z 0 

Science S 74 Ze 24 

Information Science and Technology I 99 43 28 

Engineering E 220 104 31 

Tourism i 1 2 0 

Information and Telecommunication IT 1 1 0 

Biological Science B 1 0 0 
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Table 2. Educational skills and the abbreviations Table 3. ICT classes and the number of students 
Educational skill Abbr. ICT Class Abbr. Course students 
Learning skills LRN Introduction to ICT IT Introduction 180 
Thinking skills THK Information Systems B B Introduction 80 

Wik ; Introduction to Internet , 
Inquiring skills INQ Technology N Introduction 45 
Communication skills COM iinoducdon ip Compols P Introduction 42 

Programming 

Collaboration skills CLB Basic Computer Programming BP Introduction 102 
ee saa pualding RB Introduction to Web Creation WwW Introduction 9 
Self-assessment skills SA Introduction to Data Analysis DA Introduction 64 
Evaluating other skills EO Interface Design BD Introduction 81 
Problem finding skills FND Digital Imaging DI Fundamentals 18 
Problem setting skills SET Applied Computer Programming AP Fundamentals 121 
Creative thinking skills CT Basic Web Creation BW Fundamentals 56 
Planning skills PLN Music Creation M Fundamentals 31 
Implementing skills IMP Movie Creation MV Fundamentals 18 
Continuity skills CNT Computer Networking NW Fundamentals 4+ 
Accomplishment skills ACP Applied Web Creation AW Advanced 2 
Analytical skills ANL Computer Algorithm A Advanced 40 
Mogiiyine and j MAI Computer Graphics G Advanced 19 
improvement skills 
aaa: ppopmaHon CI Software Engineering SE Advanced 3 
Decision making skills DM Operating System O Advanced 31 
Logical thinking skills LT 
Problem solving skills SLV 
Artistic skills ART 
Setting goals skills SG 
Constructing knowledge CK 
skills 


2.2 Procedure 


A computer-assisted web-interviewing (CAWI) technique was used to collect questionnaire data. Participants 
were asked to complete the online questionnaire containing the educational skills. The questionnaire and the 
study purpose information were provided for the participants by web pages. All participants joined 
voluntarily and checked the informed consent terms on the web pages. 


2.3 Questionnaire 


A questionnaire survey was conducted to analyze the 25 educational skills (Ota, 2016; Miyaji, 2011) in Table 
2 that students gained through their lessons. Students declared the educational skills themselves by the 
questionnaire. All items are rated on a nine-point Likert-type scale from | (not at all) to 9 (very high degree). 


2.4 Data Analysis 


A SOM is an efficient tool for visualization of multidimensional data. The SOM was used to visualize the 
similarity relations between educational skills based on student's consciousness. We analyze the relations of 
25 educational skills, regarding introduction, fundamentals, and advanced courses. In this analysis, 
25 educational skills (Table 2) were used as SOM variables. This study used the SOM-Toolbox (Vesanto, 
1999) in MATLAB to make and visualize the SOM for datasets. Questionnaire results for introduction, 
fundamentals, and advanced courses were used as the data sets. These data were normalized such that each 
variable has unit variance. After the SOMs were initialized and trained by principal component analysis, the 
SOM results were obtained. 
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3. RESULT 


The visualization results of the SOM are shown in Figures | to 3. There are the U-matrices (unified distance 
matrices) and the label matrices in the upper low of these figures. The other matrices are the component 
planes. In the U-matrices, the difference of reference vectors (yellow dot sequence) represents the cluster 
boundary. Each label matrix illustrates the relationship among student years, genders, ICT classes, and 
faculties, respectively. The label matrices (year, gender, class, and faculty of students) are associated with 
map units. The component planes illustrate 25 educational skills as SOM variables. In the component planes, 
yellow means higher skill rating and blue color means lower skill rating. Similar component planes are boxed 
in these figures then the planes can be classified into four or six educational skill groups. 


4. DISCUSSION 


The U-matrix of Figure | shows two cluster boundaries in the introduction course. The left-top boundary 
means students have lower educational skill values by self-evaluation and the bottom-center one means they 
have higher skill values by themselves. Therefore the students at the left-top had lower evaluation for many 
educational skills and the students at the bottom-center one had higher evaluation for many skills. 
The U-matrix of Figure 2 shows two cluster boundaries in the fundamentals course. The left-top boundary 
means students have lower educational skill values by self-evaluation and the bottom-right one means they 
have higher skill values by themselves. As in the introduction course, the students at the left-top had lower 
evaluation for many educational skills and the students at the bottom-right one had higher evaluation for 
many skills. The U-matrix of Figure 3 shows one cluster boundary in the advanced course. The left-top 
boundary means students have lower educational skill values by self-evaluation. However there is no clear 
boundary at the bottom. As in the above two courses, the students at the top had lower evaluation for many 
educational skills. Regarding educational skills, students with high ratings are placed at the bottom-left or at 
the bottom-right. 

The label matrices in Figures | to 3 show the educational skill values by self-evaluation does not depend 
on student years, genders, ICT classes, and faculties. Therefore students’ consciousness of 25 educational 
skills does not depend on these variables. 

From Figure 1, the component planes can be classified into four educational groups. Groups 11 to 41 are 
basic thinking skills, executing skills, communication skills, and creative skills, respectively. In most 
component planes, the skill evaluation is lower at the top and the evaluation is higher at the bottom. The 
evaluation values in group 31 are increasing from the left-top to the right-bottom. Meanwhile, the evaluation 
values in group 41 are increasing from the right-top to the left-bottom. Therefore, communication skills 
(group 31) are incompatible with creative skills (group 41) under students’ consciousness. From Figure 2, the 
component planes can be also classified into four educational groups. Groups 12 to 42 are basic thinking 
skills, executing skills, creative skills, and communication skills, respectively. In most component planes, the 
distributions of skill evaluations in Figure 2 are the same as Figure 1. The evaluation values in groups 12 and 
42 are increasing from the left-top to the right-bottom. Meanwhile, the evaluation values in groups 22 and 32 
are increasing from the right-top to the left-bottom. Therefore, communication skills (group 42) and basic 
thinking skills (group 12) are incompatible with creative skills (group 32) and executing skills (group 22) 
under students’ consciousness. The component planes in Figure 3 can be classified into six educational 
groups. Groups 13 to 63 are learning skills, thinking skills, problem finding skills, accomplishment skills, 
communication skills, and creative skills, respectively. For advanced course, there are three increasing 
directions in the component planes. One group is increasing from the right-top to the left-bottom. The 
appropriate groups are 13 and 23. Second group is increasing from the left-top to the right-bottom. The 
appropriate groups are 43, 53, and 63. The other group is increasing from the top to the bottom. The 
appropriate group is 33. The results of component planes for advanced course students are complicated than 
the other courses. Advanced course students might be seriously considering their educational skills because 
they had finished many ICT classes. 

Similar skills in the SOMs are thought to be highly relevant skills in the student's consciousness. It is 
difficult to teach many skills in class. Some similar skills might be able to be replaced by a few skills. There 
is a possibility that an effective lesson can be realized without imposing load to teachers and students. Even 
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now we have set educational goals and performance indicators for each lesson, but they were not analytical 
methods. Using the SOM results, we try to set more effective educational goals and performance indicators 
for each lesson. 


5. CONCLUSION 


We have propose developing a method to set key educational skills which students need to achieve for each 
class using a student self-assessment questionnaire in analytical approach. It is difficult to set key academic 
skills for class since there are little systematic methods to set them. The questionnaire survey with 25 
educational skills has been conducted to ICT classes in our university using a CAWI technique. The 
questionnaire results have been analyzed using SOMs. A SOM is used to visualize the similarity relations 
between educational skills based on student's consciousness. We have showed that 25 skills were classified 
into several skill groups of introduction, fundamentals, and advanced courses, respectively. Present and 
Future works focus on the improving class contents, the setting effective educational goals, and performance 
indicators for each lesson using SOM results. 
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Figure 1. U-matrix, the labels, and the component planes for introduction course 
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Figure 2. U-matrix, the labels, and the component planes for fundamentals course 
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Figure 3. U-matrix, the labels, and the component planes for advanced course 
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